Unifying Heterogeneous Relationships in A Single Framework for Image Segmentation

We propose a framework that can conveniently capture both the causal and non-causal relationships among random variables. The framework is formulated based on the hybrid probabilistic graphical model. It allows to model heterogeneous relationships using both the directed causal links and the undirected non-causal links. We apply this framework to image segmentation and develop a multiscale hybrid model that captures the conditional spatial relationships and the hierarchical causal relationships between regions, edges, and vertices. It also captures the causal inter-layer relationships between region labels at different scales and the enforced smoothness constraint between edge segments. We demonstrate how to factorize the joint probability distribution in such a hybrid model according to its graphical structure. Based on the factorization, we exploit the Factor Graph theory to perform the joint inference and solve the image segmentation problem.